Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Xamarin.Android.Build.Tasks] mechanism to skip ConvertResourcesCases #2348

Merged
merged 2 commits into from
Oct 31, 2018

Conversation

jonathanpeppers
Copy link
Member

@jonathanpeppers jonathanpeppers commented Oct 26, 2018

Currently, we run the ConvertResourcesCases MSBuild task for all
assemblies (on each assemby's ResolveLibraryProjectImports output).
ConvertResourcesCases is one of our slower targets.

Luckily, there are some "well-known" number of assemblies that we
could skip, so we have decided to "whitelist" certain assemblies such
as the Android support libraries, Google Play Services, Firebase, etc.

So a new @(_AndroidAssemblySkipCases) item group has been added such
as:

<_AndroidAssemblySkipCases Include="Xamarin.Android.Support.v7.AppCompat" />
<_AndroidAssemblySkipCases Include="Xamarin.Android.Support.v7.CardView" />

This <ItemGroup/> would be an indicator for ConvertResourcesCases
to just completely skip these assemblies.

Additionally, we can flat out skip aar files in the same manner.

To make this work:

  • Added support to put ITaskItem metadata in the cache file produced
    by ResolveLibraryProjectImports
  • Added item metadata for SkipAndroidResourceProcessing and
    OriginalFile.
  • ConvertResourcesCases now skips these directories and logs
    OriginalFile.
  • CollectNonEmptyDirectories needs to preserve item metadata for
    $(AndroidUseAapt2) to take advantage of the functionality.

The results appear to be well worth the effort!

Results with $(AndroidUseAapt2) enabled (note this is not the
default):

Before:
9912 ms  ConvertResourcesCases                      9 calls
2219 ms  ResolveLibraryProjectImports               1 calls

After:
  49 ms  ConvertResourcesCases                      9 calls
2185 ms  ResolveLibraryProjectImports               1 calls

Results with $(AndroidUseAapt2) disabled:

Before:
1564 ms  ConvertResourcesCases                      1 calls
1826 ms  ResolveLibraryProjectImports               1 calls

After:
  25 ms  ConvertResourcesCases                      1 calls
1685 ms  ResolveLibraryProjectImports               1 calls

This was the Xamarin.Forms-Integration project in this repo, an
initial clean build. It is basically a "Hello World" Xamarin.Forms
project. These updated numbers are from a Release build of
Xamarin.Android.

Overall this will save 1-2 seconds of ConvertResourcesCases for
default projects. This MSBuild task runs on an initial build or
incremental builds when Android resources have changed. I have gotten
slightly different numbers on the difference, each time I've compared.

There does not appear to be any noticeable slowdown in
ResolveLibraryProjectImports due to the changes.

@jonathanpeppers jonathanpeppers added do-not-merge PR should not be merged. Area: Performance Issues with performance. proposal labels Oct 26, 2018
@jonathanpeppers
Copy link
Member Author

Logs here: ConvertResourcesCases.zip

@dellis1972
Copy link
Contributor

I love this idea. It will bring major improvements for our customers.
The only downside is it will only benefit customers using the newest versions for the support libraries and the newest version of XA. The Custom Attribute will only be able to be used by support libraries built against the latest version. Not sure what would happen when a user tries to use that nuget from an older version of XA?
Can this custom attribute somewhere else that can be shared via a nuget?

Perhaps we can have a backup which uses the assembly name/info which does a similar thing so that we can bring this improvement to people using older versions of the support libraries.

@jonathanpeppers
Copy link
Member Author

If we want a build of the support libraries, they could add a file like this to their projects:

//SkipAndroidResourceProcessingAttribute.cs
[assembly: SkipAndroidResourceProcessing]

namespace Xamarin.Android.Support
{
	[AttributeUsage (AttributeTargets.Assembly)]
	public sealed class SkipAndroidResourceProcessingAttribute : Attribute
	{
	}
}

They could do this right now, even before an updated Xamarin.Android is out. (if an assembly attribute is the way we want to go)

The [assembly:LinkerSafe] attribute works this way, too. You can add an attribute named the same in a NetStandard lib, and it works because the name of the attribute matches: https://github.com/mono/linker/blob/44199a2c62ff221f92c33cbb4ee3e35588adaf4a/tuner/Mono.Tuner/CustomizeActions.cs#L47

We can for sure get this in the 28.x version of the support libs, and we might need to backport it to the version that Xamarin.Forms is using.

@jonpryor
Copy link
Member

Discussed this recently: We won't take this as-is.

The problem with an assembly-level attribute which skips all processing of the resources contained within the assembly is that it's brittle. Imagine:

  1. Dev creates a binding assembly for a .aar

  2. Dev enables [assembly: Android.SkipAndroidResourceProcessing]

  3. "Later," dev adds a custom View C# subclass and uses that View via it's C# name in a .axml file, within the same project.

    In order for this to work, <ConvertResourceCases/> must be invoked on that .axml file.

  4. App.csproj builds after (3) result in apps which throw an exception during runtime IFF the .axml from (3) is loaded (which might not be caught by unit tests!).

This is brittle. :-)

Long term, we want to be able to use this optimization, but however we provide it, it needs to not be brittle in the face of the above scenario.

@jonathanpeppers
Copy link
Member Author

jonathanpeppers commented Oct 29, 2018

We talked this through and we have a plan:

Long Term

We figure out how to do this in a nicer way, where you won't as easily be able to mess it up. Someone could add the assembly attribute without knowing it will break their layouts. This might involve redoing how __AndroidLibraryProjects__.zip works.

Short Term

We are just going to "whitelist" assemblies to skip with an <ItemGroup/> such as:

<ItemGroup>
  <_AndroidAssemblySkipCases Include="Xamarin.Android.Support.v7.CardView" />
  <_AndroidAssemblySkipCases Include="Xamarin.Android.Support.v7.AppCompat" />
  <!--Etc-->
</ItemGroup>

If we get in a tight spot, additional assemblies could be added/removed from this <ItemGroup/> without an update to Xamarin.Android.

Then our MSBuild tasks will treat @(_AndroidAssemblySkipCases) as case-insensitive list of assemblies to skip for ConvertResourcesCases.

@jonathanpeppers jonathanpeppers removed do-not-merge PR should not be merged. proposal labels Oct 29, 2018
@jonathanpeppers
Copy link
Member Author

Updated logs: ConvertResourcesCases2.zip

*******************************************
-->
<!-- As we split up/refactor this file, put new imports here -->
<Import Project="$(MSBuildThisFileDirectory)Xamarin.Android.SkipCases.projitems" />
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonathanpeppers @jonpryor will this new file be picked up by the Mac and Visx installers?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I will have to add this file in the pkg in monodroid after this is merged.

@jonathanpeppers
Copy link
Member Author

After using HashSet<string>, I don't think ResolveLibraryProjectImports has any performance slow down. I think it might actually be a little faster because I removed some LINQ, and duplicate Path.GetFullPath calls.

Latest logs (set no. 3) here: ConvertResourcesCases3.zip

@jonathanpeppers
Copy link
Member Author

Whoops, looking into the test failure

jonathanpeppers and others added 2 commits October 30, 2018 15:39
Currently, we run the `ConvertResourcesCases` MSBuild task for all
assemblies (on each assemby's `ResolveLibraryProjectImports` output).
`ConvertResourcesCases` is one of our slower targets.

Luckily, there are some "well-known" number of assemblies that we
could skip, so we have decided to "whitelist" certain assemblies such
as the Android support libraries, Google Play Services, Firebase, etc.

So a new `@(_AndroidAssemblySkipCases)` item group has been added such
as:

    <_AndroidAssemblySkipCases Include="Xamarin.Android.Support.v7.AppCompat" />
    <_AndroidAssemblySkipCases Include="Xamarin.Android.Support.v7.CardView" />

This `<ItemGroup/>` would be an indicator for `ConvertResourcesCases`
to just completely skip these assemblies.

Additionally, we can flat out skip `aar` files in the same manner.

To make this work:
- Added support to put `ITaskItem` metadata in the cache file produced
  by `ResolveLibraryProjectImports`
- Added item metadata for `SkipAndroidResourceProcessing` and
  `OriginalFile`.
- `ConvertResourcesCases` now skips these directories and logs
  `OriginalFile`.
- `CollectNonEmptyDirectories` needs to preserve item metadata for
  `$(AndroidUseAapt2)` to take advantage of the functionality.

The results appear to be well worth the effort!

Results with `$(AndroidUseAapt2)` enabled (note this is not the
default):

    Before:
    9912 ms  ConvertResourcesCases                      9 calls
    2219 ms  ResolveLibraryProjectImports               1 calls

    After:
      49 ms  ConvertResourcesCases                      9 calls
    2185 ms  ResolveLibraryProjectImports               1 calls

Results with `$(AndroidUseAapt2)` disabled:

    Before:
    1564 ms  ConvertResourcesCases                      1 calls
    1826 ms  ResolveLibraryProjectImports               1 calls

    After:
      25 ms  ConvertResourcesCases                      1 calls
    1685 ms  ResolveLibraryProjectImports               1 calls

This was the Xamarin.Forms-Integration project in this repo, an
initial clean build. It is basically a "Hello World" Xamarin.Forms
project. These updated numbers are from a `Release` build of
Xamarin.Android.

Overall this will save 1-2 seconds of `ConvertResourcesCases` for
default projects. This MSBuild task runs on an initial build or
incremental builds when Android resources have changed. I have gotten
slightly different numbers on the difference, each time I've compared.

There does not appear to be any noticeable slowdown in
`ResolveLibraryProjectImports` due to the changes.
@github-actions github-actions bot locked and limited conversation to collaborators Feb 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Area: Performance Issues with performance.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants